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MEMORANDUM  FOR  AUDITOR  GENERAL,  DEPARTMENT  OF  THE  ARMY 

AUDITOR  GENERAL,  DEPARTMENT  OF  THE  NAVY 
AUDITOR  GENERAL,  DEPARTMENT  OF  THE  AIR  FORCE 


SUBJECT:  Report  on  Followup  Review  of  the  Military  Department  Audit  Agencies  Peer 
Review  (Report  No.  D-2004-6-008) 


Government  Auditing  Standards  (GAS)  issued  in  June  2003  by  the  Comptroller 
General  of  the  United  States  require  that  organizations  conducting  Government  audits 
and  attestation  engagements  have  an  appropriate  internal  quality  control  system  in  place 
and  undergo  an  external  quality  control  review.  Organizations  conducting  audits  and 
attestation  engagements  should  undergo  an  external  quality  control  review  at  least  once 
every  3  years  by  an  organization  not  affiliated  with  the  organization  being  reviewed.  The 
external  quality  control  review  should  determine  whether  the  reviewed  audit 
organization’s  internal  quality  control  system  was  adequate  and  operating  effectively  to 
provide  reasonable  assurance  that  established  policies,  procedures,  and  applicable 
auditing  standards  were  being  followed. 

Background.  During  FY  2002,  the  Military  Department  audit  agencies 
consisting  of  the  Army  Audit  Agency  (AAA),  Naval  Audit  Service  (NAS),  and  the  Air 
Force  Audit  Agency  (AFAA)  completed  external  quality  control  reviews  of  each  other. 
We  determined  at  that  time  that  the  external  quality  control  review  results  could  be  relied 
on  to  support  the  Inspector  General  of  the  DoD  responsibilities  for  oversight  of  the 
Military  Department  audit  agencies.  For  each  of  the  reviews,  the  three  review  teams 
reported  that  the  quality  control  systems  were  designed  in  accordance  with  the  quality 
standards  established  by  the  President’s  Council  on  Integrity  and  Efficiency  (PCIE). 
Further,  the  review  teams  reported  that  each  organization’s  internal  quality  control  system 
was  operating  effectively  to  provide  reasonable  assurance  that  audit  personnel  were 
following  established  policies,  procedures,  and  applicable  reporting  standards.  In 
addition,  in  a  separate  letter  of  comments,  each  of  die  review  teams  made  observations 
and  suggested  actions  to  the  audit  agencies  to  strengthen  their  systems  of  quality  control 
on  areas  of  concern  that  did  not  warrant  inclusion  in  the  opinion  report 

In  a  November  21, 2002,  memorandum  concurring  with  the  results  of  the  peer 
review,  we  stated  that  because  some  of  the  areas  identified  as  deficiencies  were  repeat 
issues  from  the  FY  1999  peer  review,  we  would  conduct  a  followup  review  to  determine 
the  status  of  the  identified  issues  and  the  actions  taken  to  implement  the  suggested  actions 
in  the  peer  review.  . 

Review  Objective.  The  objective  of  the  review  was  to  determine  the  status  of  the 
actions  taken  to  implement  suggestions  and  recommendations  identified  in  the  FY  2002 
peer  reviews  of  the  Military  Department  audit  agencies. 

Results.  The  three  Military  Department  audit  agencies  had  taken  action  to  correct 
the  observations  made  in  the  FY  2002  peer  review.  However,  the  Military  Department 
audit  agencies  still  need  to  monitor  continuously  their  internal  quality  control  systems  and 


seek  ways  to  make  improvements,  especially  for  repeat  observations  made  in  the 
FY  1999  and  FY  2002  peer  reviews.  The  scope  and  methodology  of  the  review  are  in 
Appendix  A. 

Army  Audit  Agency.  The  AFAA  had  five  observations  concerning  the  AAA 
internal  quality  control  system  from  the  FY  2002  peer  review.  Of  the  live  observations, 
two  were  repeated  from  the  FY  1999  peer  review. 

1 .  AAA  needed  to  provide  auditors  with  a  single  reference  for  detennining 
planning  phase  requirements  and  needed  to  develop  guidance  on  reporting  on 
the  reliability  of  computer-processed  data. 

2.  AAA  could  improve  its  annual  quality  assurance  program  by  including 
broader-scoped  quality  control  review  projects  with  established  milestones  for 
each.  (Repeat  observation  from  FY  1999  peer  review) 

3.  Comments  in  quality  assurance  reports  did  not  indicate  specific  planned 
corrective  actions  and  did  not  provide  estimated  completion  dates. 

4.  Audit  teams  did  not  always  comply  with  AAA  established  quality  control 
requirements.  (Repeat  observation  from  FY  1999  peer  review) 

5 .  AAA  did  not  prepare  or  update  certain  documents  required  for  financial 
statement  audits  from  prior  year  audits. 

Guidance  Needed.  AFAA  determined  that  the  AAA  should  develop 
guidance  that  would  serve  as  a  single  reference  for  auditors  detennining  planning  phase 
requirements.  Also,  the  AAA  needed  to  develop  guidance  that  implements  the  reporting 
standard  on  the  reliability  of  computer-processed  data.  The  AAA  developed  US  AAA 
Regulation  36-02,  “Planning  the  Audit”  dated  July  14,  2003.  Regulation  36-02 
prescribed  guidance  for  planning  an  audit  engagement  within  the  AAA  including  the 
planning  process  from  identifying  future  audit  areas  to  an  entrance  conference.  In 
addition,  AAA  developed  guidance  on  the  reliability  of  computer-processed  data  that  was 
included  in  USAAA  Regulation  36-3,  “Audit  Survey  and  Execution”  dated 
October  17,  2003.  Specifically,  Appendix  G  of  Regulation  36-3  provided  guidance  on 
how  to  meet  GAS  for  perfonning  assessments  on  the  reliability  of  computer-processed 
data  and  documenting  the  assessment  results  in  the  working  papers  and  audit  reports. 

Internal  Quality  Assurance  Program.  In  relation  to  the  AAA  quality 
assurance  program,  AFAA  recommended  that  AAA  include  broader-scoped  quality 
control  review  projects  in  the  annual  quality  assurance  plan  and  establish  milestones  for 
each  quality  control  review  project.  The  AFAA  also  recommended  monitoring  the 
project  progress  against  each  milestone  and  to  the  extent  possible  assigning  one  quality 
assurance  staff  member  per  project  as  a  means  to  increase  productivity. 

Since  the  FY  2002  peer  review,  AAA  had  issued  10  internal  quality  control 
review  reports  compared  to  3  reports  issued  from  1999  through  2002.  To  add 
broader-scoped  reviews  to  the  quality  assurance  program,  AAA  had  instituted  two  types 
of  internal  quality  control  reviews:  functional  area  reviews  that  covered  multiple  program 
directors  and  post  audit  quality  reviews  that  were  comprehensive  reviews  of  individual 
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engagements  for  all  program  directors.  Four  of  the  10  reports  were  post  audit  quality 
reviews. 

Although  additional  resources  were  not  added  to  the  program,  AAA  had 
developed  a  FY  2003  through  2005  quality  assurance  plan  that  listed  the  planned 
milestones  for  each  quality  assurance  project  and  the  proposed  number  of  personnel 
assigned  to  the  project.  At  least  one  staff  member  was  assigned  to  each  project  in  the 
plan.  To  track  project  milestones  for  quality  assurance  reviews,  AAA  used  the  Army 
Audit  Agency  System  for  Information  Storage  and  Transmission  (AAAsist). 

Specifically,  AAAsist  ensures  that  employees  and  supervisors  are  aware  of  assigned  tasks 
and  due  dates  related  to  those  tasks. 

Management  Comments.  AFAA  stated  that  the  AAA  Operating  Deputy 
Auditors  General  comments  in  the  quality  assurance  reports  did  not  indicate  specific 
planned  corrective  actions  and  did  not  provide  estimated  completion  dates.  A  review  of 
the  AAA  quality  assurance  reviews  issued  since  the  FY  2002  AFAA  peer  review  showed 
that  management  comments  were  specific  to  include  corrective  actions  and  timelines  for 
completion.  Since  the  FY  2002  AFAA  peer  review,  the  AAA  also  developed  an  Excel 
spreadsheet  that  showed  each  quality  assurance  review  report,  the  recommendations,  the 
responsible  office  for  the  recommendation,  whether  there  was  concurrence  with  the 
recommendation,  and  the  implementation  date. 

Compliance  with  Quality  Control  Procedures.  Audit  teams  did  not 
always  comply  with  the  AAA  established  quality  control  requirements  consisting  of 
supervisory  review  of  working  papers,  completing  the  internal  quality  control  review 
checklists,  and  independently  referencing  the  draft  report.  According  to  AAA  personnel, 
the  three  operating  Deputy  Auditor  Generals  briefed  the  importance  of  supervisory 
reviews  during  team  and  other  meetings.  Also,  AAA  issued  guidance  to  all  supervisors 
and  managers  on  May  6,  2002,  clarifying  the  AAA  policy  on  supervisory  review  of 
working  papers.  Additional  guidance  was  issued  by  AAA  on  March  5,  2004,  that  added 
requirements  for  supervisors  to  follow  during  the  independent  report  referencing  process. 
This  guidance  emphasized  policies  and  procedures  contained  in  USAAA 
Regulation  36-85,  “Independent  Report  Referencing”  dated  October  31,  1996,  for 
independently  referencing  agency  reports.  AAA  has  also  provided  training  to  agency 
employees  on  quality  control  issues  and  has  continuously  reviewed  this  area  during  its 
quality  assurance  reviews. 

Financial  Statement  Audit  Documentation.  The  AAA  did  not  prepare 
and  update  from  prior  years  audits  certain  documents  the  Government  Accountability 
Office  (GAO)/PCIE  Financial  Audit  Manual  required  for  financial  statement  audits. 
According  to  AAA,  this  issue  had  been  put  on  hold  and  no  action  taken  because  the  AAA 
no  longer  does  financial  statement  audits. 

Naval  Audit  Service.  During  the  FY  2002  peer  review  of  the  NAS,  the  AAA 
made  two  observations  with  associated  recommendations  to  improve  the  NAS  internal 
quality  control  system.  Of  the  two  observations,  one  was  repeated  from  the  FY  1999 
peer  review. 

1 .  The  continuing  professional  education  (CPE)  statistics  in  the  NAS  training 

system  were  not  complete. 
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2.  Audit  teams  did  not  always  comply  with  quality  control  requirements  in 
the  NAS  Handbook  specifically  related  to  referencing  and  supervision. 
(Repeat  observation  from  FY  1999  peer  review) 

Continuing  Professional  Education.  AAA  determined  that  the  CPE 
statistics  in  the  NAS  training  system  were  not  complete.  To  improve  the  completeness  of 
the  NAS  CPE  statistics,  the  NAS  added  a  paragraph  to  their  Management  Handbook 
requiring  auditors  to  compare  training  records  with  data  maintained  in  the  Defense  Audit 
Management  Information  System  (DAMIS)  training  module  and  notify  training 
personnel  of  any  differences  in  the  database.  At  the  time  of  our  review,  the  NAS  was 
conducting  an  internal  quality  control  review  to  determine  whether  they  were  in 
compliance  with  continuing  professional  education  requirements  relative  to  the  GAS 
general  standard  related  to  competence. 

Compliance  with  Quality  Control  Procedures.  AAA  also  determined 
that  audit  teams  did  not  always  comply  with  quality  control  requirements  in  the  NAS 
Handbook  to  provide  reasonable  assurance  that  audits  were  conducted  in  accordance  with 
internal  policies,  procedures,  and  audit  standards.  To  increase  compliance,  the  NAS 
updated  its  referencer’s  certification  fonn  to  ensure  that  draft  reports  are  fully 
cross-referenced  before  referencing  begins  and  that  the  NAS  Assistant  Auditor  Generals 
were  apprised  of  unresolved  issues.  NAS  also  updated  the  referencer’s  guidesheet  to 
focus  on  identified  improvement  areas  and  to  require  the  use  of  more  experienced  senior 
auditors  with  good  performance  records.  In  addition,  NAS  decided  that  referencers 
would  be  given  orientation  training  before  they  referenced  reports. 

At  the  August  2002  Senior  Leadership  Conference,  the  NAS  Auditor  General, 
Deputy  Auditor  General,  and  Assistant  Auditor  Generals  decided  to  require  a 
certification  by  the  Project  Manager  and  Audit  Director  at  the  90/240  day  Auditor 
General  briefings  to  ensure  that  supervision  was  being  documented  in  accordance  with 
the  NAS  Handbook.  The  senior  leadership  of  the  NAS  also  decided  to  implement 
automated  working  papers  starting  October  1,  2002,  that  would  make  it  easier  for  the 
NAS  to  document  supervision.  In  addition,  on  October  18,  2002,  the  NAS  Policy  and 
Oversight  office  emphasized  to  all  GS-14  and  above  supervisors,  in  an  e-mail,  the  need  to 
follow  requirements  in  the  NAS  Handbook  pertaining  to  quality  control  forms, 
maintaining  complete  audit  programs,  documenting  supervision,  cross-referencing  to 
working  paper  evidence,  and  adhering  to  audit  reporting  formats.  Finally,  the  NAS 
provided  a  lessons  learned  document  in  October  2002  to  employees  that  listed  the  NAS 
Handbook  requirements  identified  by  the  AAA  peer  review  and  its  own  quality  control 
reviews  that  were  not  being  followed. 

Air  Force  Audit  Agency.  The  NAS  made  three  observations  regarding  the 
AFAA  internal  quality  control  system.  One  of  the  three  observations  was  repeated  from 
the  FY  1999  peer  review. 

1 .  AFAA  was  not  in  full  compliance  with  the  GAS  CPE  requirements. 

2.  AFAA  did  not  have  documented  policies  and  procedures  for  identifying 
when  consulting  services  were  needed  to  support  the  audit. 
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3 .  AFAA  was  not  adequately  cross-referencing  facts  and  figures  in  audit 
reports  to  supporting  working  papers.  (Repeat  observation  from  FY 
1 999  peer  review) 

Continuing  Professional  Education  Requirements.  NAS  determined 
that  the  AFAA  was  not  in  full  compliance  with  the  GAS  CPE  requirements.  Specifically, 
the  NAS  reported  that  AFAA  personnel  claimed  and  received  CPE  credit  for  training  that 
did  not  meet  CPE  guidelines,  not  all  the  AFAA  auditors  reviewed  met  the  20  hour  or 
80  hour  CPE  requirements,  and  there  was  no  AFAA  organization-wide  mechanism  to 
track  CPE  hours. 

To  improve  compliance  with  the  GAS  CPE  requirements,  the  AFAA  issued  a 
November  4,  2002,  memorandum  to  all  employees  that  provided  nine  examples  of 
programs,  activities,  subjects,  and  topics  that  did  not  qualify  for  CPE  hours.  The 
memorandum  also  encouraged  the  reading  of  the  GAO  “Interpretation  of  Continuing 
Education  and  Training  Requirements.”  The  AFAA  had  also  conducted  an  internal 
quality  control  review  to  determine  whether  the  FY  2002  NAS  peer  review  observations 
had  been  corrected.  In  a  July  8,  2004,  report,  the  AFAA  stated  that  its  review  of  training 
documentation  for  30  of  752  assigned  auditors  disclosed  all  had  met  the  20  hour  or 
80  hour  CPE  requirements  and  that  the  auditors  only  received  CPE  credit  for  allowable 
training  classes.  In  addition,  as  of  January  6,  2003,  the  AFAA  implemented/identified 
the  DAMIS  training  module  as  the  AFAA  sole  source  for  agency  personnel  to  identify 
desired  training;  complete  and  submit  the  required  forms  for  requested  training;  and 
track/certify  CPE  requirements. 

Guidance  for  using  Consultants.  NAS  also  determined  that  the  AFAA 
did  not  have  documented  policies  and  procedures  for  identifying  when  consulting 
services  were  needed  to  perform  the  required  task.  The  AFAA  added  Chapter  20,  “Use 
of  Outside  Consultants”  to  AFAA  Instruction  65-103,  “Audit  Management  and 
Administration”  dated  October  9,  2002.  This  chapter  states  that  the  audit  teams  will 
coordinate  all  decisions  to  use  consultants  with  Headquarters  AFAA  and  obtain  Auditor 
General  approval.  In  addition,  the  chapter  requires  that  the  audit  teams  identify  the  need 
for  technical  assistance  as  early  as  possible  to  allow  sufficient  lead-time  to  identify  and 
acquire  the  required  skills.  Finally,  the  chapter  provided  certain  requirements  for  audit 
teams  to  follow  before  they  engage  a  consultant. 

Cross  Referencing  of  Reports.  NAS  determined  that  facts  and  figures  in 
audit  reports  were  not  adequately  cross-referenced  to  supporting  working  papers  as 
required  by  AFAA  policies  and  procedures.  NAS  attributed  the  inadequate 
cross-referencing  to  lack  of  attention  by  audit  managers  and  lack  of  knowledge  by  the 
audit  staff  of  GAS  as  implemented  by  the  AFAA  audit  regulations. 

To  improve  cross-referencing  of  audit  reports,  the  AFAA  issued  a  memorandum 
dated  October  31,  2002,  to  all  employees  stating  that  the  requirement  to  cross-reference 
was  well  documented  by  AFAA  Instructions  65-101,  “Installation-Level  Audit 
Procedures”  and  65-102,  “Centrally  Directed  Audits,”  and  both  should  be  reviewed. 

In  addition,  AFAA  reported  in  an  internal  quality  control  review  report  dated 
July  8,  2004,  that  audit  personnel  were  adequately  cross-referencing  summary  working 
papers  to  supporting  documentation.  AFAA  evaluated  the  results  of  five  internal  quality 
control  reviews  completed  in  2003  and  2004.  The  five  internal  quality  control  reviews 
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evaluated  a  total  of  15  audit  reports.  Specifically,  AFAA  summarized  the  quality  control 
checklist  question  from  the  five  internal  quality  control  reviews  on  whether  the  audit 
manager  cross-referenced  all  facts,  figures,  and  assertions  in  the  draft  report  to  the 
summary  working  papers  and  the  summary  working  papers  to  the  supporting  working 
papers. 
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requirements  stated  in  the  quality  control  checklist  while  the  other  two  reports  partially 


Attachment 
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Appendix  A.  Scope  and  Methodology 


Review  Scope  and  Methodology.  This  review  was  performed  from  March 
through  July  2004  in  accordance  with  standards  implemented  by  the  Inspector 
General  of  the  DoD.  We  reviewed  the  Military  Department  audit  agencies  FY 
2002  peer  review  reports  and  letter  of  comments  to  determine  what  observations 
and  recommendations  were  made  for  each  of  the  audit  agencies.  We  also 
obtained  and  reviewed  policies  and  procedures,  internal  quality  assurance  reports, 
and  other  documentation  related  to  the  Military  Department  audit  agencies 
implementation  of  the  observations  and  recommendations  made  from  the 
FY  2002  peer  review.  In  addition,  we  discussed  with  the  responsible  agency 
personnel  what  action  each  of  the  Military  Department  audit  agencies  took 
regarding  the  observations  and  recommendations  made. 

Review  Limitation.  There  was  a  limitation  for  the  review  in  that  we  did  not  do 
any  testing  to  determine  whether  the  actions  taken  by  the  Military  Department 
audit  agencies  were  effective  in  improving  the  internal  quality  control  system  of 
each  of  the  respective  agencies. 

FY  2002  Peer  Review  Reports.  Copies  of  the  Military  Department  audit 
agencies  FY  2002  peer  review  reports  and  the  Inspector  General  of  the  DoD 
overall  assessment  can  be  found  at 

http://www.dodig.osd.mil/Inspections/APO/extemalreview.htm. 
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